Methods and systems for pattern characteristic detection

ABSTRACT

Disclosed are devices, systems, apparatus, methods, products, and other implementations, including a method to detect pattern characteristics in target specimens that includes acquiring sensor data for the target specimens, dividing the acquired sensor data into a plurality of data segments, and generating, by multiple neural networks that each receives the plurality of data segments, multiple respective output matrices, with each data element of the multiple respective output matrices being representative of a probability that corresponding sensor data of a respective one of the plurality of data segments includes a pattern characteristic in the target specimens. The method further includes determining by another neural network, based on the multiple respective output matrices generated by the multiple neural networks, a presence of the pattern characteristic in the target specimens.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application No.62/573,902, entitled “METHODS AND SYSTEMS FOR PATTERN CHARACTERISTICDETECTION” and filed Oct. 18, 2017, the content of which is incorporatedherein by reference in its entirety.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH

This invention was made with government support under grant 1527232awarded by the National Science Foundation. The government has certainrights in the invention.

BACKGROUND

An estimated 13% of global potential crop yield is lost to diseases eachyear, with much higher losses occurring under epidemic conditions. Forexample, northern leaf blight (NLB), also called northern corn leafblight or turcicum blight, is a fungal foliar disease of maize caused bySetosphaeria turcica (anamorph: Exserohilum turcicum). In the UnitedStates and Ontario, NLB has been growing especially severe in recentyears, with estimated yield losses rising steadily from 1.9 millionmetric tons in 2012 to 14 million metric tons in 2015. This estimatedyield loss from NLB accounted for one-fourth of all estimated yieldlosses from disease in 2015, causing an estimated economic loss of $1.9billion.

To evaluate resistance of plant germplasm and breed for improvedresistance, conventional visual assessments of disease incidence orseverity are used. However, such assessments are prone to error throughinter- and intra-rater variations, which can reduce precision andaccuracy of genetic inferences. Similar detection problems arise whenattempting to identify or assess the presence of certain patterncharacteristics (which may correspond to different abnormal conditions)in other types of specimens.

SUMMARY

Disclosed are systems, methods, and other implementations to detectpattern characteristics, which may correspond to different abnormalconditions (e.g., diseases), in various organisms (plants, animals,humans, cellular cultures of living organisms, etc.), or other specimentypes.

In some variations, a method to detect pattern characteristics in targetspecimens is provided that includes acquiring sensor data for the targetspecimens, dividing the acquired sensor data into a plurality of datasegments, and generating, by multiple neural networks that each receivesthe plurality of data segments, multiple respective output matrices,with each data element of the multiple respective output matrices beingrepresentative of a probability that corresponding sensor data of arespective one of the plurality of data segments includes a patterncharacteristic in the target specimens. The method further includesdetermining by another neural network, based on the multiple respectiveoutput matrices generated by the multiple neural networks, a presence ofthe pattern characteristic in the target specimens.

Embodiments of the method may include at least some of the featuresdescribed in the present disclosure, including one or more of thefollowing features.

The pattern characteristic may be indicative of an abnormality of thetarget specimens.

Acquiring the sensor data for the target specimens may include acquiringimage data for crop objects.

Acquiring the image data for the crop objects may include capturing oneor more of, for example, an aerial image of the crop objects, or aground-based image of the crop objects.

The image data for the crop objects may include one or more images ofcorn crops, and the presence of the pattern characteristic in the corncrops may be indicative of the presence of a northern leaf blight (NLB)disease in the corn corps.

The method may further include providing training image data to trainthe multiple neural networks and the other neural network to detect theNLB disease. Dividing the acquired sensor data into the plurality ofdata segments may include identifying a lesion with a lesion axis in oneor more images of the corn crops, defining multiple image segments ofpredetermined dimensions that are each shifted, from another of themultiple image segments, by a predetermined length of pixels and havinga center at a randomly selected location within a predetermined radiusof pixels from the lesion axis of the lesion, and rotating the each ofthe defined multiple image segments by a random rotation angle.

Generating the multiple respective output matrices may include providingthe plurality of data segments to a plurality of convolutional neuralnetwork (CNN) units, determining for a particular data segment, from theplurality of data segments, multiple probability values generatedrespectively by each of the plurality of CNN units, and writing at aparticular location, corresponding to the particular data segment, ineach of the multiple respective output matrices, respective ones of themultiple probability values determined for the particular data segment.

The acquiring, dividing, generating, and determining may be performed inreal-time.

At least some of the dividing, generating, and determining may beperformed offline.

Determining by the other neural network the presence of the patterncharacteristic in the target specimens may include determining by theother neural network the presence of the pattern characteristic in thetarget specimens further based on at least a portion of the acquiredsensor data.

Determining the presence of the pattern characteristic in the targetspecimens may include determining a percentage measure of the patterncharacteristic in the target specimens.

Acquiring the sensor data for the target specimens may include acquiringfor the target specimens one or more of, for example, visible rangeoptical data, non-visible range optical data, RF data, or environmentaldata.

In some variations, a detection system to detect pattern characteristicsin target specimens is provided. The system includes one or more sensorsto acquire sensor data for the target specimens, a controller to dividethe acquired sensor data into a plurality of data segments, and multipleneural networks, each configured to receive the plurality of datasegments and to generate multiple respective output matrices, with eachdata element of the multiple respective output matrices beingrepresentative of a probability that corresponding sensor data of arespective one of the plurality of data segments includes a patterncharacteristic in the target specimens. The system further includesanother neural network configured to determine, based on the multiplerespective output matrices generated by the multiple neural networks, apresence of the pattern characteristic in the target specimens.

Embodiments of the detection system may include at least some of thefeatures described in the present disclosure, including at least some ofthe features described above in relation to the method, as well as oneor more of the following features.

The one or more sensors may include at least one light-capture device toacquire image data for crop objects, with the image data comprising oneor more of, for example, an aerial image of the crop objects, and/or aground-based image of the crop objects.

The image data for the crop objects may include one or more images ofcorn crops, and the presence of the pattern characteristic in the corncrops may be indicative of the presence of a northern leaf blight (NLB)disease in the corn corps. The controller may further be configured toprovide training image data to train the multiple neural networks andthe other neural network to detect the NLB disease, and train themultiple neural networks and the other neural network using the trainingimage data, including to identify a lesion with a lesion axis in atleast one image, of the corn crops, from the image data, define multipleimage segments, from the at least one image, of predetermined dimensionsthat are each shifted, from another of the multiple image segments, by apredetermined length of pixels and having a center at a randomlyselected location within a predetermined radius of pixels from thelesion axis of the lesion, and rotate the each of the defined multipleimage segments by a random rotation angle.

The multiple neural networks configured to generate multiple respectiveoutput matrices may each be configured to provide the plurality of datasegments to a plurality of convolutional neural network (CNN) units,determine for a particular data segment, from the plurality of datasegments, multiple probability values generated respectively by each ofthe plurality of CNN units, and write at a particular location,corresponding to the particular data segment, in each of the multiplerespective output matrices, respective ones of the multiple probabilityvalues determined for the particular data segment.

The at least one other neural network configured to determine thepresence of the pattern characteristic in the target specimens may beconfigured to determine the presence of the pattern characteristic inthe target specimens further based on at least a portion of the acquiredsensor data.

At least one of the one or more sensors may include one or more of, forexample, a visible range optical sensor, a non-visible range opticalsensor, a RF receiver, and/or a sensor to measure one or moreenvironmental characteristics.

In some variations, an apparatus to detect pattern characteristics intarget specimens is provided. The apparatus includes means for acquiringsensor data for the target specimens, means for dividing the acquiredsensor data into a plurality of data segments, and means for generating,by multiple neural networks that each receives the plurality of datasegments, multiple respective output matrices, with each data element ofthe multiple respective output matrices being representative of aprobability that corresponding sensor data of a respective one of theplurality of data segments includes a pattern characteristic in thetarget specimens. The apparatus further includes means for determiningby another neural network, based on the multiple respective outputmatrices generated by the multiple neural networks, a presence of thepattern characteristic in the target specimens.

In some variations, a non-transitory computer readable media isprovided, with the media storing a set of instructions, executable on atleast one programmable device, to acquire sensor data for targetspecimens, divide the acquired sensor data into a plurality of datasegments, and generate, by multiple neural networks that each receivesthe plurality of data segments, multiple respective output matrices,with each data element of the multiple respective output matrices beingrepresentative of a probability that corresponding sensor data of arespective one of the plurality of data segments includes a patterncharacteristic in the target specimen. The set of instructions alsoincludes instructions to determine by another neural network, based onthe multiple respective heatmaps generated by the multiple neuralnetworks, a presence of the abnormality in the target specimens.

Embodiments of the apparatus and the computer readable media may includeat least some of the features described in the present disclosure,including at least some of the features described above in relation tothe method and to the detection system.

Other features and advantages of the invention are apparent from thefollowing description, and from the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other aspects will now be described in detail with referenceto the following drawings.

FIG. 1 is a diagram of a multi-stage system to detect patterncharacteristics in specimens.

FIG. 2 are examples of images and their associated heatmaps.

FIG. 3 is a schematic diagram of a convolutional neural networkconfiguration that may be used to perform pattern characteristicdetection.

FIG. 4 is a flowchart of an example procedure to detect patterncharacteristics in specimens.

FIG. 5 is a schematic diagram of an example computing system.

FIG. 6 provides example representative images of NLB lesions obtainedusing a handheld device, a boom device, and an unmanned aerial vehicle(UAV).

Like reference symbols in the various drawings indicate like elements.

DESCRIPTION

Described herein are systems, devices, apparatus, methods, computerprogram products, media, and other implementations to detect patterncharacteristics (e.g., corresponding to abnormal conditions, such asdiseases) in various organisms (plants, animals, humans, cellularcultures of living organisms, etc.) or other specimens' types. Inexample implementations, a system is provided that is configured toautomatically identify a pattern characteristic (which may correspond toan abnormal condition) from sensor data of a specimen (e.g., identifyingNLB lesions in field-acquired images of maize plants with highreliability). This approach uses a computational pipeline ofconvolutional neural networks (CNN) that addresses the challenges oflimited data and the myriad irregularities in the obtained data (e.g.,irregularities in images of field grown plants). In exampleimplementations to detect NLB lesions, several CNN were trained toclassify small regions of images as containing NLB lesions or not. Theirpredictions were combined into separate matrices (also referred to asheatmaps), then fed into a final, different, CNN trained to classify theentire image as containing diseased plants or not. In testing andevaluation of the implementations described herein, the system achieved96.7% accuracy on test set images not used in training. Similar systems,mounted on aerial- or ground-based vehicles, can help in automatedhigh-throughput plant phenotyping, precision breeding for diseaseresistance, and reduced pesticide use through targeted applicationacross a variety of different living organisms (plant, animal, human)and condition/disease categories.

In the example of detecting NLB, in order to reliably distinguishbetween NLB lesions and other forms of damaged or senescent leaf tissue,an NLB lesion detection system that uses convolutional neural network(CNN) machines to implement a multi-stage (e.g., three stages) imageanalysis pipeline was realized. Neural networks are in general composedof multiple layers of linear transformations (multiplications by a“weight” matrix), each followed by a nonlinear function. The lineartransformations are learned during training by making small changes tothe weight matrices that progressively make the transformations morehelpful to the final classification task. The learned multilayerprocessing of visual input that occurs in a CNN is thought to beanalogous to how the primate visual system processes information; earlystages of the networks respond to basic visual elements such as lineswhile higher levels of the networks respond to more complicated orabstract visual concepts such as object category. Convolutional layersallow a network to efficiently learn features that are invariant to anexact location in an image by applying the same learned transformationto subsections of an entire image.

Thus, with reference to FIG. 1, is a diagram of a three-stage system 100to detect abnormal conditions in organisms (or other specimens) isshown. The example system 100 depicted in FIG. 1 also includes detailsabout the training process realized to train the learning engines(machines) comprising the system 100. The training data is generallyused at some earlier time when the system is initially being configured(prior to becoming operational or being deployed to process actualdata), and during normal operation actual data is processed by thesystem 100. However, in some embodiments, output of the system 100 usingactual raw data may be used to also periodically further train thesystem 100 (e.g., by having an administrator or user confirm theaccuracy of the results obtained by the system 100 for actual data, andadjusting the configuration of the system 100 accordingly).

The system 100 implements a multi-stage processing pipeline to determinewhether a specimen, or target, has a pattern characteristic,corresponding to a condition that the system is trained to detect. Themulti-stage pipeline of the example system 100 implements a three-stagepipelines, but adding, combining, or separating different processingoperations can result in a different number of stages. In a first stage110, the system 100 implements data pre-processing and a first level ofmulti-learning engine classification performed on the pre-processeddata. Thus, to realize stage 1, the system 100 includes a datapre-processing processor/controller 112 configured to segmentmeasurement/sensor data, representative of the specimen to be evaluated(e.g., whether the specimen contains indications of abnormalities, suchas a disease afflicting it), into a plurality of data segments. Thesensor data may include image data, obtained from one or more imagecapture devices (such as a camera 102 illustrated in FIG. 1) that aremounted on an aerial platform (e.g., a drone flying over a field ofcrops), ground-based platforms (e.g., ground vehicles roaming in a fieldof crops), and/or hand-held image capture devices. The output 114 of thepre-processing processor/controller 112 includes segmented data, whichmay include substantially uniform-sized data blocks or data blocks ofvarying sizes. For example, depending on the level of informationdetermined to be contained within image regions, regions containing ahigher density of information, as determine by variability ofinformation within the regions, may be segmented into segments ofsmaller sizes. In some implementations, other types of data may be used(and pre-processed) as alternative or additional input fed to thelearning engines used by the system 100. For example, cameras configuredto obtain data in the non-visible range (e.g., infrared, UV, etc.) maybe used to obtain image data for the specimen (target object(s)).Additional examples of data that may be used to detect specific patterncharacteristics in the specimens or targets examined include RF data,environmental data (e.g., temperature, humidity, etc.), and so on.

As further illustrated in FIG. 1, the system 100 additionally includesmultiple independent CNN's 116 (depicted schematically as included in asingle module, but each CNN may be implemented as a separate module orcircuitry component) trained to detect the presence of patterncharacteristics (e.g., lesions) in small patches (segments) of theimages (each CNN may be trained using different training data sets). TheCNN's 116 receive the segmented data as part of the stage 1 processing,and generate, as part of the stage 2 (120) processing, output matrices(also referred to as heatmaps) indicating the probability that thepattern characteristic to be detected (which may correspond to theprobability of infection) is present in the regions of the images. Forexample, each data point in a generated heatmap may represent aprobability of infection, as determined by a corresponding one of theCNN's 116, that the segment/region of the image corresponding to thatheatmap data point, or pixel, contains an abnormality (e.g., infected byNLB, in the example of NLB detection). In some embodiments, the system100 may train N CNN engines, and then select from those N classifierengines a subset of the most accurate classifiers to perform the patterncharacteristic detection processing on actual data. For example, in thecase of the NLB detection, five (5) classifiers were trained, from whicha subset of three classifiers (determined to be the most accurate basedon some pre-defined error criterion) were selected to perform the actualdetection processing.

Training of the CNN classifiers of stage 1 may be performed as follows.Training image data may be broken up into small segments (e.g., by thecontroller 112), such as segments of, for example, 224 by 224 pixels.For training purposes, training personnel may mark or annotate atraining data set (in some examples, the annotation task may beoutsourced to third-party personnel through a platform such asMechanical Turk (MTurk)). In the example of training the system 100 todetect NLB, an image 106 may be annotated by a trainer to identifyregions with NLB lesions, and further identify or define a lesion axis.In some embodiments, a lesion axis may be determined based on automaticfiltering/processing procedures (which may also be implemented using thecontroller 112) to identify structures that are similar to lesion shapes(e.g., identifying contours of candidate lesions and determining iftheir morphology is substantially close to, in a mathematical sense, totypical contours of an actual lesion). In some embodiments,identification of lesion may be performed by an independent learningengine configured (through previous training) to identify candidatelesions. Alternatively, a hybrid process that combines human annotationof images (to identify lesions) and automated identification of lesionsmay be used. For example, automatically identified lesions may beassigned a confidence metric, and lesions with a confidence metric belowa certain high reference threshold (and optionally above a low referencethreshold/level, that together define a range) may be passed to a humanassessor that can annotate the image. Images in which identified lesionare above the high reference threshold may be deemed to include theidentified lesion, while automatically identified lesions with aconfidence score/metric below the low reference threshold may be deemedto not include a lesion (or the image may be discarded).

Training image segments (process to be annotated with identified lesionsand their lesion axes) may then be generated by moving down the lengthof the lesion, e.g., 10 pixels at a time and centering the targetsegment in a randomly chosen location ±15 pixels from a major axis ofthe lesion. A random rotation may then be applied to the image beforethe final segmentation, which allows, for example, six slightlydifferent images to be produced from the same area. Non-lesion trainingsegments may also be produced from the non-infected images in a similarfashion, i.e., by randomly rotating segments drawn from the entireimage.

After training one network on the smaller image patches, that networkmay be used to select images to augment the data set (a technique calledhard negative mining). Images of non-infected plants may be broken upinto, for example, regions of 224 by 224 pixels using a sliding windowapproach with a pre-determined step size of, for example, 55. Becausethe original training images may be randomly rotated and nudged up ordown, only a small fraction (e.g., fewer than 0.1% in some embodiments)of these newly generated segments could be identical to the firsttraining set. These image segments can be fed into the trained neuralnetwork. Image segments that are incorrectly classified as containinglesions may then added to the training set. Versions of these images mayalso be added be the training set after they are flipped horizontallyand vertically.

Image data used may be collected using different devices and/orplatforms that provide different viewing perspectives and resolutions ofthe specimens analyzed. In an example training and prediction procedurethat was implemented during testing and evaluation of the systems andmethods described herein, some of the image data used was collected by ahandheld device with a digital single-lens reflex (DSLR) camera withangle and field of view variable. Another set of image data wascollected using a DSLR was mounted on a boom (˜5 m from ground, nadirview), and a further set of test image data was collected by an unmannedaerial vehicle (UAV) images to obtain aerial photographs of 4 acres ofvarious (nadir view). FIG. 6 provides example representative images ofNLB lesions obtained, respectively, through the handheld device (seeimage 610), the boom device (see image 620), and the UAV (see image630). For all handheld images, lesions were marked down the main axiswith a line. For boom images, images were split in four and lesions weremarked using an ImageJ macro. The handheld image set was split intothree groups: 1) a training set that was used to train the initialnetworks, 2) a validation set that was used to choose threewell-performing networks for input to stage 3 of the processingpipeline, and 3) a test set used for testing accuracy of the final CNN.In the example implementation, training of CNNs was performed in threestages: stage i) training set images subdivided and CNNs trained onsmaller sub-images, stage ii) CNNs predicted lesion presence/absencealong a sliding window, generating heat maps of confidence that a lesionis present at a given location, and stage iii) A third CNN was trainedon heat maps output by CNNs A/B/C from Stage 2. in this exampleimplementation, three distinct CNNs were trained on the training set.The three CNNs differed in architecture, balance of lesion/non-lesionimages, and source of non-lesion images. Some non-lesion images weregenerated by hard negative mining (iteratively adding non-lesions imagesthat a CNN initially misclassified as containing lesions to the trainingset). More particularly, a first CNN (CNN A) used equal parts lesion andnon-lesion images (with only negatively mined non-lesion images beingused). A second CNN (CNN B) used a 6:1 ratio of non-lesion images tolesion images. For that CNN, original non-lesion images and negativelymined images were used. A third CNN (CNN C) used a 6:1 ratio ofnon-lesion images to lesion images. For the third CNN, originalnon-lesion images and negatively mined images used.

With continued reference to FIG. 1, the multiple trained CNN's 116 thatreceive the pre-processed data 114 in stage 1 are configured to generateoutput matrices (heatmaps) 122 in a second stage (stage 2) 120. One setof heatmaps is produced from each CNN trained to classify small patchesof the images (or other types of data) in stage 1. An output matrix, orheatmap, may comprise a single row of probability elements (i.e., theoutput matrix may have dimensions of 1×m, where m is the number ofcolumns), while in some other embodiments the output matrix may havenrows of probability elements (i.e., a matrix of dimensions n×m). In someimplementations, a sliding window approach may be used with a step sizeof, for example, 30, to feed sections of the larger images through eachof the neural networks. The output may be recorded for eachregion/location in the images, and can be interpreted as the probabilitythat each section contains a lesion. Each probability is assembled intoa matrix in its appropriate place so as to represent the output of anetwork for each area or region of the image.

FIG. 2 includes examples of images and their associated heatmaps (i.e.,output matrices). Particularly, images 200 and 210 are example images ofcrops containing lesions, while images 220 and 230 are example images ofcrops not containing lesions. In the example of FIG. 2, three heatmapsfor each of the images 200, 210, 220, and 230 were generated using threedifferent CNN's (which may correspond to the CNN A, B, C illustrated inFIG. 1). For example, an image 202 corresponds to a heatmap for theimage 200 generated using a first CNN, an image 204 corresponds to aheatmap for the image 200 using a second CNN (B), and an image 206corresponds to a heatmap generated using a third CNN. The other heatmapsprovided in FIG. 2 similarly include the heatmaps generated by the samethree CNN's for the images 210, 220, and 230. As shown, heatmaps areshaded to indicate the output score representative of a probability ofan image segment containing a lesion. A probability of ‘1’ (one)corresponds to white while pixel, and a probability of ‘0’ (zero) isrepresented by black. Intermediate probability values are represented byshades of gray.

Turning back to FIG. 1, in a third stage (130) implemented by thedetection systems described herein, at least one other trained learningengine 132, such as another CNN (separate from the multi-CNN's used togenerate the heatmaps from the subdivided images), is configured toclassify each entire image as containing or not containing abnormalspecimen (e.g., containing or not containing infected leaves in the NLBexample). In some embodiments, neural networks were trained on variouscombinations of the heatmaps. The heatmaps were “stacked” on top of eachother, each occupying a different channel of the input. Input from threeheatmaps of the same underlying image, for example, would occupy threechannels just as a color image would be composed of three channels(e.g., red, green, and blue). They may be flipped horizontally andvertically and rotated during training to augment their number. In someembodiments, various combinations of the heatmap sets were used fortraining and a selection of three heatmap set was made based onvalidation set performance. In some embodiments, the heatmaps generated(at stage 2) may be used (e.g., by a learning engine such as thelearning engine 132) to provide an estimate of the percentage of thedata that corresponds to the pattern characteristic that is beingdetected (e.g., leaf area inn a captured image that is suspected ofbeing diseased). Thus, in such embodiments, instead of merely providinga yes/no output, the system 100 is configured to compute a quantitativemeasure in relation to the input data (e.g., percentage of diseasedcrops).

FIG. 3 is a schematic diagram of a convolutional neural network (CNN)configuration 300 that may be used to perform pattern characteristicdetection (e.g., to detect presence of crop diseases in crops based onimages of the crops). This configuration may be used for each of thelearning engines implemented in the pipeline of FIG. 1. The exampleembodiments of the CNN configuration 300 includes a first convolutionlayer 310 comprising, for example, 64 filters (more or fewer filters maybe used). An input image 302 activates the set of 64 filters in theconvolutional layer 310, with each filter associated with a givenweight-describing parameters. In some embodiments, every data segments(partitioned or divided during a pre-processing stage) is processed togenerate a resultant data point produced by the different filters of theconvolutional layer 310.

Coupled to the convolutional layer 310 is a pooling layer 320 configuredto decrease the resolution of the resultant data from the layer 310, andto allowing for more filters in subsequent (downstream) layers.Accordingly, additional convolutional layers 330 and 350, locateddownstream from the first convolution layer 310, are activated byhigher-level features and information, with pooling layers (such as thepooling layers 340 and 360) disposed at locations immediately following(in the configurational hierarchy) the convolution layers 330 and 350respectively, to thus allow for expanding number of filters. Weights areprogressively optimized using backpropagation, which estimates howchanges in filter weights would affect the final level of error. Theweight values of the filters may be determined using one or moreoptimization techniques, such as gradient descent. A final logistic unit370 integrates information from top layer(s) to classify the image.

With reference next to FIG. 4, a flowchart of an example procedure 400to detect pattern characteristics (corresponding to differentabnormalities) in target specimens is shown. The procedure 400 includesacquiring 410 sensor data for the target specimens. As noted, the targetspecimens may be any of variety of organisms, including plants, animals(marine or land), humans, sections of cellular tissue (e.g., canceroustissue from any type of organism), etc. The sensor data acquired mayinclude optical data (visible and/or non-visible range), RF data(electromagnetic data in different RF bands), environmental data (e.g.,temperature, humidity, and so on), etc. In some of the exampleembodiments described herein, acquiring sensor data for the targetspecimens may include acquiring image data for crop objects. Such cropobjects may include corn crops, and in some examples, the abnormalitythat is to be detected may be a northern leaf blight (NLB) disease.Capturing image data for the crop objects may be performed by capturingone or more of an aerial image of the crop objects (e.g., throughdrone-mounted camera(s)), and/or a ground-based image of the cropobjects (e.g., by a handheld device, by an image-capture device attachedto a boom, by an autonomous ground roaming vehicle, etc.)

The procedure 400 further includes dividing 420 the acquired sensor data(be it image data, or some other sensor data) into a plurality of datasegments. As discussed, such data segmentation may be performed by apre-processing processor/controller (such as the controller 112 ofFIG. 1) as one of the initial stages of a multi-stage pipelineimplementation for the systems/methods described herein.

With continued reference to FIG. 4, the procedure 400 also includesgenerating 430, by multiple neural networks that each receives theplurality of data segments, multiple respective output matrices(heatmaps), with each data element of the multiple respective outputmatrices being representative of a probability that corresponding sensordata of a respective one of the plurality of data segments includes apattern characteristic in the target specimens. As noted, a heatmapgenerally has smaller dimensions than the data processed by the neuralnetworks, with each element (e.g., a pixel, or a matrix element) holdinga value indicative of the probability that a data segment (e.g., an i×jimage segment) contains the pattern characteristic (associated with someabnormality, such as NLB disease) being checked for. In someembodiments, the neural networks may be implemented as convolutionalneural network (CNN) devices/units. In such example embodiments,generating the multiple respective output matrices may include providingthe plurality of data segments to a plurality of convolutional neuralnetwork (CNN) devices/units, determining for a particular data segment,from the plurality of data segments, multiple probability valuesgenerated respectively by each of the plurality of CNN units, andwriting at a particular location, corresponding to the particular datasegment, in each of the multiple respective output matrices, respectiveones of the multiple probability values determined for the particulardata segment.

With the various neural network devices (a subset of such neuralnetworks may be selected from a larger set of networks based, forexample, on accuracy or some other performance criterion) havinggenerated the respective output matrices, the procedure 400 alsoincludes determining 440 by another neural network (which also may be aCNN device, or some other type of neural network), based on the multiplerespective output matrices generated (e.g., at 430) by the multipleneural networks, a presence of the pattern characteristic in the targetspecimens. Thus, the other neural network device (which may be similarto the neural network 132 of FIG. 1) may be trained to recognize, basedon multiple output matrices it receives as input from the multipleneural networks of the earlier stage (such as the neural networks 112 ofFIG. 1, or some other intermediate neural network device(s)), whetherthe specimen for which the sensor data was acquired has the patterncharacteristic (abnormality) the system is configured to detect orrecognize. In some variations, determining by the other neural networkthe presence of the pattern characteristic in the target specimens mayinclude determining by the other neural network the presence of thepattern characteristic in the target specimens further based on at leasta portion of the acquired sensor data. In some embodiments, determiningthe presence of the pattern characteristic in the target specimens mayinclude determining a quantitative measure (percentage) representativeof the presence of the pattern characteristic in the target specimens.

As noted, the various neural networks (be it the networks 112 or thenetwork 132) generally need to be trained to properly recognize certaindesired features or patterns. For example, in embodiments in which thesensor data acquired is image data for corn crops, and the patterncharacteristic is one corresponding to an NLB disease abnormality, theprocedure 400 may further include providing training image data to trainthe multiple neural networks and the other neural network to detect theNLB disease, and training the multiple neural networks and the otherneural network using the training image data. The training may includeidentifying a lesion with a lesion axis in at least one image of thecorn crops from the image data, defining multiple image segments ofpredetermined dimensions that are each shifted, from another of themultiple image segments, by a predetermined length of pixels and havinga center at a randomly selected location within a predetermined radiusof pixels from the lesion axis of the lesion, and rotating the each ofthe defined multiple image segments by a random rotation angle.

In some embodiments, the operations of acquiring (at 410), dividing (at420), generating (at 430), and determining (at 440) may be performed inreal-time. Alternatively, in some embodiments, at least some of theoperation of dividing (at 420), generating (at 430), and determining (at440) may be performed offline (e.g., at a remote server to whichacquired data is provided or communicated).

Performing the various operations described herein may be facilitated bya controller system (e.g., a processor-based controller system).Particularly, at least some of the various devices/systems describedherein, including any of the neural network devices, the pre-processingcontroller, a remote server or device that performs at least some of thedetection operations (such as those described in relation to FIG. 4),etc., may be implemented, at least in part, using one or moreprocessor-based devices.

Thus, with reference to FIG. 5, a schematic diagram of a computingsystem 500 is shown. The computing system 500 includes a processor-baseddevice (also referred to as a controller device) 510 such as a personalcomputer, a specialized computing device, and so forth, that typicallyincludes a central processor unit 512, or some other type of controller.In addition to the CPU 512, the system includes main memory, cachememory and bus interface circuits (not shown in FIG. 5). Theprocessor-based device 510 may include a mass storage element 514, suchas a hard drive (realize as magnetic discs, solid state (semiconductor)memory devices), flash drive associated with the computer system, etc.The computing system 500 may further include a keyboard 516, or keypad,or some other user input interface, and a monitor 520, e.g., an LCD(liquid crystal display) monitor, that may be placed where a user canaccess them.

The processor-based device 510 is configured to facilitate, for example,the implementation of detection of pattern characteristics in targetspecimens based on the procedures and operations described herein. Thestorage device 514 may thus include a computer program product that whenexecuted on the processor-based device 510 causes the processor-baseddevice to perform operations to facilitate the implementation ofprocedures and operations described herein. The processor-based devicemay further include peripheral devices to enable input/outputfunctionality. Such peripheral devices may include, for example, aCD-ROM drive and/or flash drive (e.g., a removable flash drive), or anetwork connection (e.g., implemented using a USB port and/or a wirelesstransceiver), for downloading related content to the connected system.Such peripheral devices may also be used for downloading softwarecontaining computer instructions to enable general operation of therespective system/device. Alternatively or additionally, in someembodiments, special purpose logic circuitry, e.g., an FPGA (fieldprogrammable gate array), an ASIC (application-specific integratedcircuit), a DSP processor, etc., may be used in the implementation ofthe system 500. Other modules that may be included with theprocessor-based device 510 are speakers, a sound card, a pointingdevice, e.g., a mouse or a trackball, by which the user can provideinput to the computing system 500. The processor-based device 510 mayinclude an operating system, e.g., Windows XP® Microsoft Corporationoperating system, Ubuntu operating system, etc.

Computer programs (also known as programs, software, softwareapplications or code) include machine instructions for a programmableprocessor, and may be implemented in a high-level procedural and/orobject-oriented programming language, and/or in assembly/machinelanguage. As used herein, the term “machine-readable medium” refers toany non-transitory computer program product, apparatus and/or device(e.g., magnetic discs, optical disks, memory, Programmable Logic Devices(PLDs)) used to provide machine instructions and/or data to aprogrammable processor, including a non-transitory machine-readablemedium that receives machine instructions as a machine-readable signal.

In some embodiments, any suitable computer readable media can be usedfor storing instructions for performing theprocesses/operations/procedures described herein. For example, in someembodiments computer readable media can be transitory or non-transitory.For example, non-transitory computer readable media can include mediasuch as magnetic media (such as hard disks, floppy disks, etc.), opticalmedia (such as compact discs, digital video discs, Blu-ray discs, etc.),semiconductor media (such as flash memory), electrically programmableread only memory (EPROM), electrically erasable programmable read onlyMemory (EEPROM), etc.), any suitable media that is not fleeting or notdevoid of any semblance of permanence during transmission, and/or anysuitable tangible media. As another example, transitory computerreadable media can include signals on networks, in wires, conductors,optical fibers, circuits, any suitable media that is fleeting and devoidof any semblance of permanence during transmission, and/or any suitableintangible media.

To test and evaluate the implementations described herein, severalexperiments were conducted. In one experiment, images of NLB-infectedand noninfected leaves were taken with a Canon EOS Rebel or Sony a6000camera by hand on dates ranging from 28 to 78 days post-inoculation(DPI). Altogether, 1,834 images were taken over eight dates. In total,38 images were excluded due to poor quality. The images were firstclassified by presence or absence of any visible lesions in the image.Following this, all visible lesions were marked with a line down themain axis of the lesion using the annotation features of the Bisqueimage-processing platform hosted on CyVerse (formerly iPlant). Thus, atotal of 1,796 images of maize leaves, which included 1,028 images ofNLB-infected leaves and 768 images of noninfected leaves were used. The1,796 images obtained were randomly divided such that 70% of the imageswere used for training (i.e., to fit the models), 15% for validation(i.e., to estimate prediction error for model and hyperparameterselection), and 15% for testing (i.e., to assess error of the finalchosen model). All choices involving network architecture and alltraining were done without consideration of the test set, which was onlyused at the end to assess the performance of the final, full system.

There were two image sizes: 6,000 by 4,000 pixels and 5,184 by 3,456pixels. The images of infected leaves were annotated for NLB lesionswith 6,931 lines, or an average of 6.7 lines/image. This was slightlyhigher than the number of lesions, because lesions that curved due toleaf curvature were annotated with multiple lines. Other sources ofsenesced leaf tissue were present in both the inoculated andnon-inoculated trial. These mainly included physical damage, naturalsenescence of lower leaves, nitrogen deficiency, feeding from corn fleabeetles, and other foliar diseases, particularly northern corn leafspot. Lesions on inoculated plants were typical of those present innatural infections, and lesion color and shape were generally comparablebetween the inoculated and non-inoculated plantings.

The three chosen CNN classifiers of the stage 1 system (similar to thesystem 100 of FIG. 1) achieved accuracies of 81%, 95%, and 94% inclassifying small image segments from the validation set. The finalstage 3 network (e.g., the CNN 132) was trained on various combinationsof the heat maps produced using networks that were trained in stage 1.When those heat maps were combined, the stage 3 network achieved 97.8%accuracy on the validation set. The validation set was used to guidetraining by helping to make the choice of architecture of the finalsystem as well as determining which heat maps were used and what valueswere chosen for various hyperparameters in the neural networks.

In order to have an accurate understanding of how the system wouldperform on new images, a test set of 272 images was left completelyunused throughout the entire training procedure. Because the test setdid not influence the system in any way, an estimate of error based onit was expected to be unbiased. On this test set, the network achievedan accuracy (number of correctly classified images divided by totalnumber of images) of 96.7%, a precision of 96.8% (number of truepositives [i.e., truly diseased] divided by the number of true positivesplus false positives), a 97.4% recall (number of true positives dividedby the number of true positives plus the number of false negatives), andan F1 score (2×precision×recall, all divided by precision plus recall)of 0.971.

Several issues created challenges to successfully classifying theimages. The first was the small number of images to train on; successfulapplication of deep learning techniques typically involves largertraining sets, on the order of tens of thousands. Another factorcontributing to the task's difficulty lay in the nature of the imagesthemselves. Many types of dead leaf tissue, including naturalsenescence, can closely resemble NLB lesions to both a CNN and theuntrained eye. Variation in lighting, a common issue for images taken inthe field, also presented problems. Areas of shadow or, conversely,bright light appearing on a leaf were often mistaken by networks earlyin training as lesions; they were well represented in the falsepositives found during hard negative mining. Leaves in the background,dead leaf tissue on the ground, senescing leaves in the lower canopy,and insects also presented challenges.

One significant benefit of the three-stage pipeline was the ability ofthe system to make use of the full-resolution images. Compared withscaling the images down, cropping them into smaller full-resolutionsections in the first stage allowed the network to make use of thefine-grained detail that distinguishes an NLB lesion from other brownspots or dead tissue. On their own, though, the small segmentpredictions actually presented a problem because of their sheer number.Because the heat maps (matrices of 126 by 193 values) contained scoresfor 24,318 such segments, even a highly accurate classifier would havemany errors in its scores for an entire image. The best stage 1 networkachieved an accuracy of 94%; thus, it would be expected that over 1,000incorrectly classified segments in every one of the heat maps. However,the use of the stage 3 classifier (e.g., the CNN 132 of FIG. 1)mitigates this problem, as the stage 3 classifier learned how to combineall of the local segment scores, including inaccurate ones, into aglobal classification, achieving 96.7% accuracy on whole images.

The testing and evaluation of the implementations described herein alsoincluded a project to collect image data in order to develop arepository of images that could be used for further development ofanalysis and diagnostic procedures (e.g. based on computer vision anddeep learning approaches) for agriculture and other subject matterareas. In that project, image data from several platforms and angles wasacquired to help develop a system for real-time monitoring andphenotyping of NLB in maize fields using drones equipped with CNNs. Theresulting data set exceeds 18,000 maize plant images annotated with morethan 100,000 NLB lesions, which is the largest collection of images forany one plant disease.

More particularly, the data collected in this project included threeimage sets and their accompanying annotations. All images were taken infield trials of maize that had been inoculated with Setosphaeriaturcica, the causal agent of NLB. All trials were planted at CornellUniversity's Musgrave Research Farm in Aurora, N.Y. The trials includedmaize hybrids from “The Genomes to Fields Initiative,” arranged intwo-row plots with a length of 5.64 m and inter-row spacing of 0.76 m.There was a 0.76 m alley at the end of each plot. The trials wererainfed and managed with conventional maize cultivation practices.Plants were inoculated at the V5-V6 stage with both a liquid suspensionof S. turcica (isolate NY001) spores and sorghum grains colonized by thefungus. The first image set, namely, the “handheld set,” was taken byhand. The second, namely the “boom set,” was taken by mounting thecamera on a 5 m boom. This boom held the remotely triggered camera abovethe canopy with nadir view. The third, the “drone set,” was taken bymounting the camera on a DJI Matrice 600 sUAS. The drone was flown at analtitude of 6 m and a velocity of 1 m/s, and images were captured withnadir view every 2 s.

For the handheld and boom sets, images were checked manually to ensurethe image was in focus and otherwise adequate. For the drone set, imageswith a low total length of edges (as reported by canny edge detection)were filtered out, in order to remove blurry images. Images were thendiscarded during annotation if they were out of focus or otherwiseunacceptable. In each image, lesions were annotated by one of two humanexperts, as denoted in the annotation files. Annotators drew a line downthe main axis of each lesion visible in the image, stretching down theentire length of the lesion. If a lesion appeared bent or curved fromthe camera's perspective, two or more intersecting annotation lines weredrawn to form an angle or arc as needed. In the handheld set, this wasdone with the markup tools in Bisque. In the boom and drone sets, thesesteps were done using a custom ImageJ macro. Endpoint coordinates ofeach annotation line were recorded, in this project, in ‘.csv’ datafiles, each corresponding to a single data set. Images with 0 values forall four endpoint coordinates had no visible lesions.

The number of images and annotation lines were as follows:

-   -   Handheld: 1787 images, 7669 annotations.    -   Boom: 8766 images, 55,919 annotations.    -   Drone: 7669 images, 42,117 annotations.

Some boom images were ¼ slices of larger images, as a wider field ofview made it difficult to annotate the entire image at once.Accordingly, these images were assigned file names with suffixes such as‘img01_00.jpg’, ‘img01_01.jpg’, and so on.

The implementations described herein combine the output of differentclassifiers to derive a final classification output (e.g., lesion orno-lesion). This ensemble approach shows improved performance over asingle classifier. The best result was achieved with a combination ofthree classifiers in the stage 1 part of the network, so that the systembenefited from this effect. However, in some evaluation trials, evenwhen only one network's heat maps was used in the third stage there wasstill observed significant improvement over the initial, baselinenetwork, which took as input scaled-down versions of the full images.Therefore, the three-stage system's improved performance was not onlydue to the multi-classification configuration, but also due to otherfactors. For instance, neural network performance is greatly affected bythe amount of data available for training. Because the first stage ofthe pipeline was trained on small sections of the images instead of thefull image, the training set size was effectively increased, at leastfor the networks in stage 1. One lesion might be broken up into manysmall regions, for example. The need to break up the images and processthem in stages arose, in part, because of memory constraints (it isdifficult to feed reasonably-sized batches of full-resolution imagesinto a CNN comprising a reasonable number of convolutional filters perlayer). Conceivably, making end-to-end training of the entire pipelinefrom full-resolution images to final classification, it is possible thatperformance could be further improved.

On-site diagnosis requires the ability to detect the presence of diseasesymptoms in images that are not ideal and that contain many potentiallyconfounding factors. Because machine learning techniques typically donot perform as well on data that are significantly different from thaton which they were trained, it is recommended that that classifiers betrained on images taken in similar conditions in the field.

The implementations described herein can detect the presence or absenceof a disease in an image, information most readily used for estimatingdisease incidence. This may be useful for growers looking for earlydetection or breeders evaluating incubation period for a given disease.In some embodiments, some of the implementations described herein may beconfigured to quantify disease severity. This information could beextracted from the heat maps used for detection. Factors such asdistance from the camera, leaf angle, and the amount of non-plantfeatures within the image present challenges for calculating theproportion of diseased versus healthy plant tissue within an image.

While the implementations discussed herein were described in relation toan application to detect NLB, these implementations and approaches canbe applied to other plant diseases, as well as to other organisms (e.g.,determine existence of cancer tissue in a specimen, determine animal andhuman diseases and abnormalities, etc.) and/or other fields and subjectmatter. As noted, in some embodiments, the systems described herein maybe mounted on an aerial vehicle. Coupled with autonomous orsemiautonomous navigation, sUAS (small unarmed aircraft systems)platforms could provide measures of disease in the field with greateraccuracy and the same or less human input than current visual diseasemonitoring. Such a system could perform the detection operation eitherin real-time, or offline (in the latter case, a moving platform, such asa drone, would be equipped with sensors to collect data that could bestored and later downloaded to a remote device, or transmit the data tothe remote device). In a production setting, the detection system couldbe coupled with real-time variable-rate fungicide applicators. Suchapplicators feed measured crop parameters into a decision support systemto gauge the required fungicide dosage. This limits fungicideapplication rates in areas where it is less needed (e.g., based on adetermination that the incidence of a particular disease, in aparticular area where the crops are growing, is below some threshold)with the dual benefit of reducing fungicide usage and runoff and savingmoney for growers.

Unless defined otherwise, all technical and scientific terms used hereinhave the same meaning as commonly or conventionally understood. As usedherein, the articles “a” and “an” refer to one or to more than one(i.e., to at least one) of the grammatical object of the article. By wayof example, “an element” means one element or more than one element.“About” and/or “approximately” as used herein when referring to ameasurable value such as an amount, a temporal duration, and the like,encompasses variations of ±20% or ±10%, ±5%, or +0.1% from the specifiedvalue, as such variations are appropriate in the context of the systems,devices, circuits, methods, and other implementations described herein.“Substantially” as used herein when referring to a measurable value suchas an amount, a temporal duration, a physical attribute (such asfrequency), and the like, also encompasses variations of ±20% or ±10%,±5%, or +0.1% from the specified value, as such variations areappropriate in the context of the systems, devices, circuits, methods,and other implementations described herein.

As used herein, including in the claims, “or” as used in a list of itemsprefaced by “at least one of” or “one or more of” indicates adisjunctive list such that, for example, a list of “at least one of A,B, or C” means A or B or C or AB or AC or BC or ABC (i.e., A and B andC), or combinations with more than one feature (e.g., AA, AAB, ABBC,etc.). Also, as used herein, unless otherwise stated, a statement that afunction or operation is “based on” an item or condition means that thefunction or operation is based on the stated item or condition and maybe based on one or more items and/or conditions in addition to thestated item or condition.

Although particular embodiments have been disclosed herein in detail,this has been done by way of example for purposes of illustration only,and is not intended to be limiting with respect to the scope of theappended claims, which follow. Features of the disclosed embodiments canbe combined, rearranged, etc., within the scope of the invention toproduce more embodiments. Some other aspects, advantages, andmodifications are considered to be within the scope of the claimsprovided below. The claims presented are representative of at least someof the embodiments and features disclosed herein. Other unclaimedembodiments and features are also contemplated.

What is claimed is:
 1. A method to detect pattern characteristics intarget specimens, the method comprising: acquiring sensor data for thetarget specimens; dividing the acquired sensor data into a plurality ofdata segments; generating, by multiple neural networks that eachreceives the plurality of data segments, multiple respective outputmatrices, with each data element of the multiple respective outputmatrices being representative of a probability that corresponding sensordata of a respective one of the plurality of data segments includes apattern characteristic in the target specimens; and determining byanother neural network, based on the multiple respective output matricesgenerated by the multiple neural networks, a presence of the patterncharacteristic in the target specimens; wherein the method furthercomprises: providing training image data to train the multiple neuralnetworks and the other neural network to detect northern leaf blight(NLB) disease in corn crops; and training the multiple neural networksand the other neural network using the training image data, including:identifying a lesion with a lesion axis in at least one image of thecorn crops from the image data; defining multiple image segments ofpredetermined dimensions that are each shifted, from another of themultiple image segments, by a predetermined length of pixels and havinga center at a randomly selected location within a predetermined radiusof pixels from the lesion axis of the lesion; and rotating the each ofthe defined multiple image segments by a random rotation angle.
 2. Themethod of claim 1, wherein the pattern characteristic is indicative ofan abnormality of the target specimens.
 3. The method of claim 1,wherein acquiring sensor data for the target specimens comprises:acquiring image data for crop objects.
 4. The method of claim 3, whereinacquiring the image data for the crop objects comprises: capturing oneor more of: an aerial image of the crop objects, or a ground-based imageof the crop objects.
 5. The method of claim 3, wherein the image datafor the crop objects comprises one or more images of the corn crops, andwherein the presence of the pattern characteristic in the corn crops isindicative of the presence of the northern leaf blight (NLB) disease inthe corn crops.
 6. The method of claim 1, wherein generating themultiple respective output matrices comprises: providing the pluralityof data segments to a plurality of convolutional neural network (CNN)units; determining for a particular data segment, from the plurality ofdata segments, multiple probability values generated respectively byeach of the plurality of CNN units; and writing at a particularlocation, corresponding to the particular data segment, in each of themultiple respective output matrices, respective ones of the multipleprobability values determined for the particular data segment.
 7. Themethod of claim 1, wherein the acquiring, dividing, generating, anddetermining are performed in real-time.
 8. The method of claim 1,wherein at least some of the dividing, generating, and determining areperformed offline.
 9. The method of claim 1, wherein determining by theother neural network the presence of the pattern characteristic in thetarget specimens comprises: determining by the other neural network thepresence of the pattern characteristic in the target specimens furtherbased on at least a portion of the acquired sensor data.
 10. The methodof claim 1, wherein acquiring the sensor data for the target specimenscomprises: acquiring for the target specimens one or more of: visiblerange optical data, non-visible range optical data, RF data, orenvironmental data.
 11. The method of claim 1, wherein determining thepresence of the pattern characteristic in the target specimenscomprises: determining a percentage measure of the patterncharacteristic in the target specimens.
 12. A detection system to detectpattern characteristics in target specimens, the system comprising: oneor more sensors to acquire sensor data for the target specimens; acontroller to divide the acquired sensor data into a plurality of datasegments; multiple neural networks, each configured to receive theplurality of data segments and to generate multiple respective outputmatrices, with each data element of the multiple respective outputmatrices being representative of a probability that corresponding sensordata of a respective one of the plurality of data segments includes apattern characteristic in the target specimens; and at least one otherneural network configured to determine, based on the multiple respectiveoutput matrices generated by the multiple neural networks, a presence ofthe pattern characteristic in the target specimens; wherein thecontroller is further configured to: provide training image data totrain the multiple neural networks and the other neural network todetect northern leaf blight (NLB) disease in corn crops; and train themultiple neural networks and the other neural network using the trainingimage data, including to: identify a lesion with a lesion axis in atleast one image, of the corn crops, from the image data; define multipleimage segments, from the at least one image, of predetermined dimensionsthat are each shifted, from another of the multiple image segments, by apredetermined length of pixels and having a center at a randomlyselected location within a predetermined radius of pixels from thelesion axis of the lesion; and rotate the each of the defined multipleimage segments by a random rotation angle.
 13. The detection system ofclaim 12, wherein the one or more sensors comprise at least onelight-capture device to acquire image data for crop objects, wherein theimage data comprises one or more of: an aerial image of the cropobjects, or a ground-based image of the crop objects.
 14. The detectionsystem of claim 13, wherein the image data for the crop objectscomprises one or more images of the corn crops, and wherein the presenceof the pattern characteristic in the corn crops is indicative of thepresence of the northern leaf blight (NLB) disease in the corn corps.15. The detection system of claim 12, wherein the multiple neuralnetworks configured to generate multiple respective output matrices areeach configured to: provide the plurality of data segments to aplurality of convolutional neural network (CNN) units; determine for aparticular data segment, from the plurality of data segments, multipleprobability values generated respectively by each of the plurality ofCNN units; and write at a particular location, corresponding to theparticular data segment, in each of the multiple respective outputmatrices, respective ones of the multiple probability values determinedfor the particular data segment.
 16. The detection system of claim 12,wherein the at least one other neural network configured to determinethe presence of the pattern characteristic in the target specimens isconfigured to: determine the presence of the pattern characteristic inthe target specimens further based on at least a portion of the acquiredsensor data.
 17. The detection system of claim 12, wherein at least oneof the one or more sensors comprises one or more of: a visible rangeoptical sensor, a non-visible range optical sensor, a RF receiver, or asensor to measure one or more environmental characteristics.
 18. Anon-transitory computer readable media storing a set of instructions,executable on at least one programmable device, to: acquire sensor datafor target specimens; divide the acquired sensor data into a pluralityof data segments; generate, by multiple neural networks that eachreceives the plurality of data segments, multiple respective outputmatrices, with each data element of the multiple respective outputmatrices being representative of a probability that corresponding sensordata of a respective one of the plurality of data segments includes apattern characteristic in the target specimen; and determine by anotherneural network, based on the multiple respective heatmaps generated bythe multiple neural networks, a presence of the abnormality in thetarget specimens; wherein the set of instructions comprises one or morefurther instructions, executable on the at least one programmabledevice, to further: provide training image data to train the multipleneural networks and the other neural network to detect northern leafblight (NLB) disease in corn crops; and train the multiple neuralnetworks and the other neural network using the training image data,including to: identify a lesion with a lesion axis in at least oneimage, of the corn crops, from the image data; define multiple imagesegments, from the at least one image, of predetermined dimensions thatare each shifted, from another of the multiple image segments, by apredetermined length of pixels and having a center at a randomlyselected location within a predetermined radius of pixels from thelesion axis of the lesion; and rotate the each of the defined multipleimage segments by a random rotation angle.